From Simple Stochastic Games to Bisimulation Pseudometrics on Markov Decision Processes

In this paper we investigate the complexity of computing bisimulation pseudometrics on Markov decision processes (MDPs). Our first main result is that such pseudometrics can be computed in the complexity class PPAD. We show that another well-known problem in PPAD—computing the value of a simple stochastic game (SSG)— can be reduced in logarithmic space to the problem of computing the bisimulation pseudometric on a given MDP. In the other direction, we reduce the problem of computing the bisimulation pseudometric to that of computing the value of an SSG. This reduction uses a construction similar to the classical attacker-defender game for bisimulation in the non-probabilistic case, and works in polynomial time for MDPs of a fixed branching degree. Finally, we investigate whether the above bound on the branching degree can be dropped, relating it to the question of whether there is a family of polynomial size SSGs that solve the linear assignment problem.